Skip to main content

What is software testing and types of testing?

Software Testing: Definition and Importance​

Software testing is a systematic process of evaluating a software system or its components to identify gaps, errors, or missing requirements compared to the expected requirements. It is a critical quality assurance activity within the software development lifecycle that aims to ensure that the software:

  1. Meets specified requirements
  2. Works as expected under various conditions
  3. Is free from defects and bugs
  4. Is reliable, maintainable, and usable

Testing is essential because it:

  • Identifies and helps fix defects early in the development cycle, reducing costs
  • Improves software quality and reliability
  • Enhances user experience and satisfaction
  • Reduces maintenance costs and technical debt
  • Mitigates risks and ensures compliance with regulations

Types of Software Testing​

Software testing can be classified in multiple ways based on different criteria. The main categories include:

1. Based on Knowledge of Internal Structure​

a. White Box Testing​

White box testing (also known as clear box or glass box testing) examines the internal structure, design, and coding of the software. The tester has complete knowledge of the internal workings of the application.

Characteristics:

  • Tests internal paths, code statements, branches, and conditions
  • Requires programming knowledge and access to the source code
  • Focuses on strengthening security, improving design, and optimizing code

Examples:

  • Statement Coverage: Ensures each line of code is executed at least once
  • Branch Coverage: Tests each branch in decision points
  • Path Coverage: Tests all possible paths through the code

Techniques:

// Code to test
public int calculateDiscount(int purchaseAmount) {
    int discount = 0;
    
    if (purchaseAmount > 1000) {
        discount = 100;
    } else if (purchaseAmount > 500) {
        discount = 50;
    } else {
        discount = 10;
    }
    
    return discount;
}

// Test cases for branch coverage
@Test
public void testLargeDiscount() {
    assertEquals(100, calculateDiscount(1500)); // Branch 1
}

@Test
public void testMediumDiscount() {
    assertEquals(50, calculateDiscount(750));   // Branch 2
}

@Test
public void testSmallDiscount() {
    assertEquals(10, calculateDiscount(200));   // Branch 3
}

b. Black Box Testing​

Black box testing evaluates the functionality of the software without knowledge of its internal structure or coding. The tester views the software as a "black box" and is only concerned with inputs and outputs.

Characteristics:

  • Tests only the functionality, not the implementation details
  • Doesn't require programming knowledge
  • Focuses on user requirements and specifications

Examples:

  • Equivalence Partitioning: Divides input data into valid and invalid partitions
  • Boundary Value Analysis: Tests boundaries between partitions
  • Decision Table Testing: Tests combinations of inputs

Techniques:

Example of Boundary Value Analysis for an age input field (valid range: 18-65):

Test Cases:
1. Age = 17 (just below lower boundary) -> Expected: Invalid
2. Age = 18 (lower boundary) -> Expected: Valid
3. Age = 19 (just above lower boundary) -> Expected: Valid
4. Age = 64 (just below upper boundary) -> Expected: Valid
5. Age = 65 (upper boundary) -> Expected: Valid
6. Age = 66 (just above upper boundary) -> Expected: Invalid

c. Gray Box Testing​

Gray box testing is a combination of white box and black box testing. The tester has partial knowledge of the internal structure to design better test cases.

Characteristics:

  • Limited knowledge of internal structure
  • Combines functional and structural testing approaches
  • Often used for integration testing and security testing

Examples:

  • Database testing while understanding table structures
  • Testing error handling when knowing error boundary conditions

2. Based on Testing Level​

a. Unit Testing​

Unit testing focuses on testing individual components or modules of the software in isolation. This is typically the first level of testing and is often performed by developers.

Characteristics:

  • Tests the smallest testable parts of an application
  • Often automated using frameworks like JUnit, NUnit, or PyTest
  • Finds bugs early in development
  • Makes code more reliable and easier to refactor

Example:

// Code to test
public class Calculator {
    public int add(int a, int b) {
        return a + b;
    }
}

// Unit test
@Test
public void testAddition() {
    Calculator calc = new Calculator();
    assertEquals(5, calc.add(2, 3));
    assertEquals(0, calc.add(-2, 2));
    assertEquals(-5, calc.add(-2, -3));
}

b. Integration Testing​

Integration testing verifies that different modules or services work well together. It focuses on the interfaces between components and their interaction with each other.

Characteristics:

  • Tests component interactions
  • Identifies interface defects
  • Can be approached in several ways (Big Bang, Top-down, Bottom-up, etc.)

Types:

  • Big Bang Integration: All components are integrated at once
  • Top-down Integration: Higher-level modules are tested first, using stubs for lower modules
  • Bottom-up Integration: Lower-level modules are tested first, using drivers for higher modules
  • Sandwich/Hybrid Integration: Combines top-down and bottom-up approaches

Example:

// Integration test for a user authentication system
@Test
public void testUserLoginAndDatabase() {
    UserService userService = new UserService(databaseConnection);
    boolean result = userService.authenticateUser("username", "password");
    assertTrue(result);
    
    // Verify in the database that the login was recorded
    UserActivity activity = databaseConnection.getLastActivity("username");
    assertEquals("LOGIN", activity.getType());
}

c. System Testing​

System testing evaluates the complete and integrated software to verify that it meets specified requirements. It tests the software as a whole, in an environment that resembles the production environment.

Characteristics:

  • Tests the entire application's functionality
  • Verifies the software against system requirements
  • Tests both functional and non-functional aspects
  • Usually performed in a test environment similar to production

Types:

  • Functionality testing
  • Performance testing
  • Usability testing
  • Recovery testing
  • Security testing

Example:

System Test Case: Online Shopping Checkout Process

Preconditions:
- User is logged in
- Items are in the shopping cart

Test Steps:
1. Navigate to shopping cart
2. Verify items and quantities are correct
3. Click "Proceed to Checkout"
4. Enter shipping information
5. Select shipping method
6. Enter payment information
7. Review order
8. Confirm order

Expected Results:
- Order confirmation page appears
- Confirmation email is sent
- Inventory is updated
- Order is recorded in the database

d. Acceptance Testing​

Acceptance testing determines if the software meets the user requirements and is ready for delivery. It is typically performed by end-users or clients.

Characteristics:

  • Final testing phase before release
  • Validates the software against business requirements
  • Determines if the software is acceptable for delivery

Types:

  • User Acceptance Testing (UAT): Testing by end-users
  • Alpha Testing: Internal testing in a controlled environment
  • Beta Testing: External testing by a limited number of real users
  • Business Acceptance Testing: Verifies business workflows

Example:

Acceptance Test Case: Expense Report Submission

Scenario: Employee submits an expense report

Steps:
1. Login as an employee
2. Navigate to expense management
3. Create a new expense report
4. Add three different expenses with receipts
5. Submit the report for approval

Expected Results:
- Report is successfully submitted
- Manager receives notification
- Employee can track status of the report

3. Based on Testing Approach​

a. Manual Testing​

Manual testing is performed by a human tester who executes test cases without using automated tools. The tester follows a written test plan and compares expected and actual results.

Advantages:

  • Provides human insight and intuition
  • Better for exploratory, usability, and ad-hoc testing
  • No script maintenance required
  • Better for short-term projects

Disadvantages:

  • Time-consuming
  • Prone to human error
  • Difficult to repeat precisely
  • Limited for performance and load testing

b. Automated Testing​

Automated testing uses specialized tools and scripts to execute tests automatically. This approach is efficient for repetitive tasks and regression testing.

Advantages:

  • Faster execution
  • Reusable test scripts
  • Consistent and repeatable
  • Better for regression and performance testing
  • Reduces human error

Disadvantages:

  • Initial setup cost and time
  • Requires programming skills
  • Script maintenance overhead
  • May miss certain issues that humans would notice

Example:

// Automated UI test using Selenium
@Test
public void testLoginFunctionality() {
    WebDriver driver = new ChromeDriver();
    driver.get("https://example.com/login");
    
    driver.findElement(By.id("username")).sendKeys("testuser");
    driver.findElement(By.id("password")).sendKeys("password123");
    driver.findElement(By.id("loginButton")).click();
    
    // Assert that login was successful
    WebElement welcomeMessage = driver.findElement(By.id("welcome"));
    assertEquals("Welcome, testuser!", welcomeMessage.getText());
    
    driver.quit();
}

4. Based on Testing Time/Phase​

a. Static Testing​

Static testing examines the software artifacts (requirements, design documents, code) without executing the code. It helps identify defects early in the development cycle.

Techniques:

  • Code reviews
  • Walkthrough
  • Inspections
  • Static code analysis

Example:

// Code with potential issues for static analysis
public void processData(String input) {
    if (input.length() > 0) {  // Potential NullPointerException if input is null
        // Process data
    }
    
    File file = new File("output.txt");
    FileWriter writer = new FileWriter(file);  // Resource leak: writer not closed
    writer.write(input);
}

// Static analysis tools would flag:
// 1. Potential NullPointerException
// 2. Resource leak (unclosed FileWriter)

b. Dynamic Testing​

Dynamic testing involves executing the code and validating the outputs against expected outcomes. It examines the behavior of the software during runtime.

Types:

  • Unit testing
  • Integration testing
  • System testing
  • Performance testing

5. Based on Testing Objective/Focus​

a. Functional Testing​

Functional testing verifies that each function of the software operates according to the functional requirements. It focuses on the "what" the system does.

Types:

  • Smoke Testing: Basic tests to verify the critical functionality
  • Sanity Testing: Quick evaluation to verify that specific functionality works
  • Regression Testing: Re-testing after changes to ensure existing features still work
  • Localization Testing: Tests product adaptation to specific locales

Example:

Functional Test Case: User Registration

Steps:
1. Navigate to registration page
2. Enter username, email, and password
3. Click "Register" button

Expected Result:
- User account is created
- Confirmation email is sent
- User can log in with new credentials

b. Non-Functional Testing​

Non-functional testing verifies aspects of the software that are not related to specific functions or features but rather to operational aspects. It focuses on the "how" the system works.

Types:

  1. Performance Testing: Evaluates system responsiveness and stability under various load conditions

    • Load Testing: Tests behavior under expected and peak load conditions
    • Stress Testing: Tests behavior beyond normal operating capacity
    • Endurance/Soak Testing: Tests behavior under sustained load over time
    • Spike Testing: Tests behavior when load increases suddenly

  2. Security Testing: Identifies vulnerabilities and ensures data protection

    • Penetration testing
    • Vulnerability scanning
    • SQL injection testing
    • Cross-site scripting (XSS) testing

  3. Usability Testing: Evaluates how user-friendly the software is

    • User experience testing
    • UI/UX testing
    • Accessibility testing

  4. Compatibility Testing: Ensures software works across different environments

    • Browser compatibility
    • Operating system compatibility
    • Device compatibility
    • Database compatibility

  5. Reliability Testing: Verifies system stability and consistency

    • Recovery testing
    • Failover testing
    • Disaster recovery testing

Example:

Performance Test Case: Website Response Time

Objective: Verify the website responds within 2 seconds under normal load

Setup:
- 1000 concurrent users
- Various user scenarios (browsing, searching, purchasing)
- 30-minute test duration

Metrics to Measure:
- Response time
- Throughput
- Error rate
- CPU and memory usage

Acceptance Criteria:
- 95% of requests complete in under 2 seconds
- Error rate less than 1%
- Server CPU usage below 80%

6. Other Important Testing Types​

a. Regression Testing​

Regression testing ensures that recent code changes haven't adversely affected existing features. It involves re-running tests to verify that previously working functionality still works.

Approaches:

  • Retest all
  • Regression test selection
  • Test case prioritization

b. Exploratory Testing​

Exploratory testing is a simultaneous learning, test design, and test execution approach where the tester explores the application to identify defects that might be missed by scripted testing.

Characteristics:

  • Emphasizes personal freedom and responsibility
  • Simultaneously involves learning, test design, and test execution
  • Relies on tester's creativity and intuition

c. Smoke Testing​

Smoke testing is a preliminary test to verify that the basic and critical functionalities of the software work before proceeding with more extensive testing.

Characteristics:

  • Quick and shallow
  • Covers major functionality
  • Determines if the build is stable enough for further testing

d. A/B Testing​

A/B testing compares two versions of a webpage or app against each other to determine which one performs better for a given conversion goal.

Process:

  1. Create two variants (A and B)
  2. Split the audience randomly
  3. Analyze which variant performs better
  4. Implement the winning variant

e. Accessibility Testing​

Accessibility testing ensures that the application is usable by people with disabilities (visual, hearing, motor, or cognitive).

Guidelines:

  • Web Content Accessibility Guidelines (WCAG)
  • Section 508 compliance

Testing Process and Lifecycle​

The testing process typically follows these stages:

  1. Test Planning: Define the scope, approach, resources, and schedule for testing
  2. Test Analysis: Study the requirements to identify testable aspects
  3. Test Design: Create test cases and procedures
  4. Test Environment Setup: Prepare the testing environment
  5. Test Execution: Run the tests and report defects
  6. Test Closure: Evaluate test completion criteria and prepare test summary reports
┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│  Requirements │      │Test Planning & │      │ Test Design & │
│  Analysis     │─────►│ Test Analysis  │─────►│ Development   │
└───────────────┘      └───────────────┘      └───────┬───────┘
                                                      │
                                                      ▼
┌───────────────┐      ┌───────────────┐      ┌───────────────┐
│  Test Closure │◄─────│  Defect       │◄─────│ Test Execution│
│  & Reporting  │      │  Tracking     │      │ & Evaluation  │
└───────────────┘      └───────────────┘      └───────────────┘

Test Documentation​

Effective software testing requires proper documentation:

  1. Test Plan: Outlines the strategy, resources, schedule, and deliverables
  2. Test Case: Detailed steps, expected results, and actual results
  3. Test Scenario: High-level description of what to test
  4. Test Script: Detailed instructions for automated tests
  5. Defect Report: Detailed description of a discovered issue
  6. Test Summary Report: Overview of testing activities and results

Example Test Case Format:

Test Case ID: TC-001
Test Case Name: Verify User Login
Priority: High
Prerequisite: User is registered

Test Steps:
1. Navigate to login page
2. Enter valid username
3. Enter valid password
4. Click login button

Expected Result:
User should be logged in and redirected to dashboard

Actual Result:
[To be filled during execution]

Status: [Pass/Fail]

Notes:
[Any observations or additional information]

Testing Principles​

Seven fundamental principles guide the practice of software testing:

  1. Testing shows the presence of defects, not their absence: Testing can show that defects exist but cannot prove there are no defects.

  2. Exhaustive testing is impossible: Complete testing of all combinations is impractical; risk analysis and priorities are essential.

  3. Early testing saves time and money: Testing activities should start as early as possible in the development lifecycle.

  4. Defects cluster together: A small number of modules usually contain most defects.

  5. Beware of the pesticide paradox: Repeating the same tests will eventually stop finding new bugs; tests need to evolve.

  6. Testing is context dependent: Different systems require different testing approaches.

  7. Absence of errors is a fallacy: Finding and fixing defects doesn't help if the system is unusable or doesn't fulfill user needs.

Test-Driven Development (TDD)​

Test-Driven Development is a development approach where tests are written before the code. The process follows a Red-Green-Refactor cycle:

  1. Red: Write a failing test
  2. Green: Write the minimum code to make the test pass
  3. Refactor: Improve the code while keeping tests passing

Example:

// Step 1: Write a failing test
@Test
public void testAddItem() {
    ShoppingCart cart = new ShoppingCart();
    cart.addItem("Apple", 1.99);
    assertEquals(1, cart.getItemCount());
    assertEquals(1.99, cart.getTotal(), 0.001);
}

// Step 2: Write minimum code to make it pass
public class ShoppingCart {
    private List<Item> items = new ArrayList<>();
    
    public void addItem(String name, double price) {
        items.add(new Item(name, price));
    }
    
    public int getItemCount() {
        return items.size();
    }
    
    public double getTotal() {
        return items.stream().mapToDouble(Item::getPrice).sum();
    }
}

// Step 3: Refactor as needed while keeping tests passing

Continuous Testing in DevOps​

Continuous Testing is an approach that embeds testing throughout the CI/CD pipeline, providing immediate feedback on risks:

  • Shift-Left Testing: Moving testing earlier in the development process
  • Automated Testing: Using automation to test continuously
  • Test Environment Management: On-demand test environments
  • Service Virtualization: Simulating unavailable systems
  • Test Data Management: Providing appropriate test data
┌─────────┐   ┌─────────┐   ┌─────────┐   ┌─────────┐   ┌─────────┐
│         │   │         │   │         │   │         │   │         │
│ Develop ├──►│  Build  ├──►│   Test  ├──►│ Deploy  ├──►│ Operate │
│         │   │         │   │         │   │         │   │         │
└─────────┘   └─────────┘   └─────────┘   └─────────┘   └─────────┘
    ▲             ▲             ▲             ▲             ▲
    │             │             │             │             │
    └─────────────┴─────────────┴─────────────┴─────────────┘
                   Continuous Testing

Testing Metrics and Measurements​

Key metrics for evaluating testing effectiveness include:

  1. Test Coverage: Percentage of code/requirements tested

    • Statement coverage
    • Branch coverage
    • Path coverage
    • Requirement coverage

  2. Defect Metrics:

    • Defect density (defects per size unit)
    • Defect detection percentage
    • Defect leakage (defects missed during testing)
    • Defect removal efficiency

  3. Test Execution Metrics:

    • Test pass/fail rate
    • Test execution time
    • Test automation percentage
    • Test effectiveness

  4. Product Quality Metrics:

    • Mean Time Between Failures (MTBF)
    • Mean Time To Repair (MTTR)
    • Customer satisfaction

Conclusion​

Software testing is an essential part of software development that ensures quality, reliability, and user satisfaction. By implementing various testing types at different levels and phases of development, organizations can:

  • Identify and fix defects early
  • Reduce development costs and time-to-market
  • Improve product quality and user experience
  • Mitigate risks and ensure regulatory compliance

As software systems become more complex and release cycles shorten, the importance of comprehensive testing strategies, automation, and continuous testing becomes even more critical to delivering high-quality software products.